Direct manipulation interfaces are popular because they allow computer users to import intuitions from their real world experience. Most standard programming languages, such as Fortran, are very different from everyday languages, such as English. Developers of computer user interfaces have recognized the advantages of taking the physical world metaphor seriously. HyperCard.sup..RTM. (trademark of Apple Computer, Inc.), for example, allows the user not only to use the computer in the ordinary sense of the word (that is, by causing the computer to perform a predefined task such as accessing a database, file, stack, etc., stored in a memory) but also to modify its computer user interface by editing the pictorial images ("icons") and texts, which indicate action-initiating areas displayed on the screen, by causing the computer to perform an interface-modifying operation.
Action-initiating areas related to such predetermined tasks are often called a button or a menu item, depending upon how they appear and/or how they function. An action-initiating area can also be as simple as a line segment or a graphical element. In what follows, action-initiating areas of all shapes and designs may be comprehensively referred to as a "display object" or simply as an "object." In other words, a display object is whatever is displayed on the screen and is associated with a certain predefined task to be performed by the computer as explained above. The user can cause the computer to perform a desired task by selectively indicating on the screen a particular display object (such as a button or a textual menu item) associated with the desired task and sending an activating signal to the computer. For this purpose, a mouse may be used, which not only can move a position-indicating symbol (also known as a pointer) on the display screen but also can be "clicked" by depressing a switch (also known as a "button", albeit a physical one) on the mouse.
When a mouse is used to "press" a button display object, for example, to thereby cause the computer to perform the predefined task associated with the button, it may be said that the button has been "used." As mentioned above, however, a modern interface is adapted not only to be "used" in the sense of the word explained above, but also to allow the user to modify it by editing its attributes. When the user wants to modify attributes of a button, for example, the user is not necessarily wanting to "use" it so as to cause the computer to perform the predefined task associated with this button. Instead, the user merely wants to "mention" the button to the computer such that desired modifications of attributes ("editing") will be effected on the desired button (or that the computer will perform an interface-modifying operation). To "use" a button and to "mention" (or "edit") a button are basically different types of operations, and prior art programming made it troublesome to get back and forth between these two types or modes of operation. This has often been referred to as the use-mention (or use-edit) problem by the developers of interfaces and programs therefor.
In the modal approach of HyperCard.sup..RTM., as in most prior art systems, the user is required to decide whether functionalities of the interface are to be modified but not used or, instead, to be used but not modified. As shown schematically in FIG. 5 by way of a simplified event loop flowchart, the user is required to choose (or remain in) the use-only mode (called "Browse Mode") if functionalities of the interface are going to be invoked (used) rather than modified. In this mode of operation, all display objects (such as buttons) that can be used appear on the screen (or at least as many as the screen size allows), as shown in part in FIG. 6, and the system moves in a loop within this mode. In FIG. 6, icons 11, 12, 13 and 14, named respectively "Appointments", "Addresses", "Puzzle" and "Train Set" represent buttons for separately accessing databases, files, stacks, etc., of these names, and the triangle 16 and the two rectangular textual buttons 17 and 18 are each for causing the computer to perform a certain predefined task. Button Mode and Field Mode shown in FIG. 5 are two of the basic edit-only modes wherein specified functionalities are merely mentioned for editing but not used or invoked. There are other edit-only modes for carrying out various specific interface-modifying operations, but they are not included in FIG. 5 in order to simplify the explanation. The user, while in Browse Mode, can enter Button Mode for modifying attributes of a button, Field Mode for modifying attributes of a field (i.e., an area containing textual information) or any of the other edit-only modes, for example, by pressing a corresponding button. These attributes usually include such things as editing the name of the button or field, editing styles and/or fonts of the button or field name, editing the display image of the button or field (e.g., changing, moving, resizing, etc.), deleting the button or field, obtaining further information about the button or field, etc.
In this manner, HyperCard.sup..RTM., as in most prior art systems, allows display objects (such as buttons and menu items) to be directly manipulated in either of the two major modes, one associated with use and the other with mention. However, one major limitation of these prior art systems is that the user is required to decide in which of the modes these objects are to be manipulated. Thus, repeatedly crossing the modal barrier can become tedious when, for example, the user wants to modify an object and then try it out to see the result. These prior art systems require the user to continually remember what mode he/she is in. Although the graphical representation of the cursor may be changed from one mode to another, the user still tends to forget what mode he/she is in. If the user tries to use a button while in the mode for mentioning, for example, the computer will not respond with an execution of the predefined task, and the user may think that the button or system is broken.
One approach in the prior art to alleviate the user confusion created when a system is in the mention-only mode and the user merely wants to perform a use function is to essentially hobble the system, from the user perspective, so that the system is only allowed to operate in a use-only mode. In this way, the user need not worry about what mode the system is operating in because the system is functionally hobbled to be in only one mode, the use-only mode. Of course, this hobbling, by definition, eliminates even the possibility of mentioning or editing the display objects and as such is a more limited or restricted, albeit simplified, user interface system.
According to one of the prior art methods addressed to this problem (Randall B. Smith, David Ungar and Bay-Wei Chang: SIGCHI `91 Workshop on Computer Languages for Programming User Interface Software), special places or indicators ("affordances") are displayed on or adjacent buttons. Each affordance is associated with a different way in which the button can be modified. When the user wants to modify a button in a certain way by "mentioning" the button to the computer, a position indicator, such as a cursor or pointer, is moved on the display screen (say, by means of a mouse) to the particular affordance corresponding to the desired way of modifying the button. If an activation signal is sent to the computer (say, by clicking on the mouse) with the indicator pointing to the desired affordance, it is the affordance, and not the button itself, that is selected or "pressed." Thus, the computer does not perform the task associated with the button, but instead modifies the button by changing one of its attributes.
There are problems, however, with this prior art affordance display approach. On the one hand, because the affordances were always displayed in this prior art approach, and because the affordances must not be allowed to interfere with the use of the button, this required the affordances to be relatively small (which thus further limited the quantity and range of affordances as well as each affordances' ability to visually indicate or depict its functionality). On the other hand, the affordances could not be so small, nor the display of the affordances so cluttered, that they became difficult for the user to find or remember what each affordance does. In addition, there may be situations where it is not desirable to have additional areas reserved for the affordances at all such as, for example, for aesthetic reasons associated with reducing visual clutter or for ease of use reasons associated with making the computer user interface more simplified albeit more limited in capability. Without any hobbled use-only mode in this prior art affordance system, and without any ability to selectively display affordances on a button-by-button (object-by-object) basis, it was not possible to prevent display of all affordances nor to show a multiplicity of affordances, complex or otherwise. In summary, affordances, as hitherto considered, are awkward to use and lack the degree of functionality desired by the present system.
According to another prior art approach, a palette of various editing "tools" was provided for carrying out different modifying operations on buttons. This required the user to select one of these tools for each kind of operation to be carried out. Once selected, each of these tools was adapted to stick to the cursor and be moved around so that the user could then "place" the editing tool over the object to be edited and thereby perform the desired editing operation. This method was cumbersome because the user had to switch tools between every two editing operations that were to be performed and was less intuitive because there was less association between the editing tools and the object to be edited.